How Experience of the Body Shapes Language about Space

Open-ended language communication remains an enormous challenge for autonomous robots. This paper argues that the notion of a language strategy is the appropriate vehicle for addressing this challenge. A language strategy packages all the procedures that are necessary for playing a language game. We present a specific example of a language strategy for playing an Action Game in which one robot asks another robot to take on a body posture (such as stand or sit), and show how it effectively allows a population of agents to self-organise a perceptually grounded ontology and a lexicon from scratch, without any human intervention. Next, we show how a new language strategy can arise by exaptation from an existing one, concretely, how the body posture strategy can be exapted to a strategy for playing language games about the spatial position of objects (as in “the bottle stands on the table”). 1 Language Games for Embodied Agents Over the past decade we have been investigating in our group through what mechanisms open-ended language can be grounded in situated embodied interactions by performing computer simulations and doing experiments with physical autonomous robots [Steels, 2001]. Empirical research on natural dialog [Garrod and Doherty, 1994] has shown that language is not a static system. Instead language must be viewed as a complex adaptive system that is shaped and reshaped by its users, even in the course of a single dialog, in order to remain maximally adaptive to the expressive needs of the community, while at the same time maximising communicative success and minimising cognitive effort [Hopper, 1987]. We use a whole systems approach, as pioneered in behavior-based robotics [Steels and Brooks, 1994], meaning that all aspects of the problem, from perception and action, to categorisation, meaning selection, parsing, and production, as well as learning and embodied interaction, are operationalised and integrated into a single system, so that none of the components needs to be perfect but the whole system is stronger and more reliable than each of the parts taken separately. Our methodology has crystallized around a set of central concepts. The first one is the notion of a language game [Steels, 1995]. A language game is a routinised interaction between a speaker and a listener out of a population whose members have regular interactions with each other. Each individual agent in the population can be both speaker and hearer. The game has a non-linguistic goal, which is some situation that speaker and hearer want to achieve cooperatively. For example, the speaker may want to draw the attention of the hearer to some object in the world. Speaker and hearer can use bits of language but they can also use pointing gestures and non-verbal interaction, and there is a shared common ground so that not everything needs to be said explicitly. A typical example of a language game is the Color Naming Game, which is a game of reference, where the speaker uses a color to draw the attention of the hearer to an object in the world [Steels and Belpaeme, 2005]. Humans users are able to play thousands of different games and even a single sentence may involve different language games simultaneously. For example, if somebody says “give me the red block” there is both an Action Game (asking somebody else to do something) and a Reference Game (drawing attention to an object in the world). The second central concept in our work is that of a language strategy. A language strategy is a set of procedures that will allow members of a community to become and remain effective in playing a particular language game. It includes not only the interaction script, the turn-taking, joint attention, and other non-linguistic aspects of the game, but also procedures for perceiving, conceptualising, and acting upon reality, for producing and parsing utterances, for interpreting meaning back into the world, and for acquiring both the concepts, words and grammatical constructions needed in the game. In addition, a language strategy contains procedures for diagnosing failure in an interaction, for repairing the failure and for aligning conceptual and linguistic inventories so that speakers and hearers get maximally attuned to each other. When agents start to exercise a language strategy through a series of games in concrete situated interactions, each agent progressively builds a particular language system, i.e. a particular ontology, lexicon and grammar. For example, a strategy for playing the Color Naming Game will 14 Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-09)